Information processing apparatus, information processing method and computer program product

ABSTRACT

According to an embodiment, an information processing apparatus includes a first image acquisition unit, a first information acquisition unit, a reception unit, a generation unit, and a writing unit. The first image acquisition unit is configured to acquire a first image obtained by imaging by an imaging unit. The first information acquisition unit is configured to acquire any of multiple pieces of information capable of being associated with the first image. The reception unit is configured to receive a first input. The generation unit is configured to associate the first image acquired in the first image acquisition unit with the information piece acquired in the first information acquisition unit to generate an association data piece when the reception unit receives the first input. The writing unit is configured to write the association data piece into a storage unit.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based upon and claims the benefit of priority fromJapanese Patent Application No. 2013-076979, filed on Apr. 2, 2013; theentire contents of which are incorporated herein by reference.

FIELD

Embodiments described herein relate generally to an informationprocessing apparatus, an information processing method, and a computerprogram product.

BACKGROUND

Conventionally, there is known a technique of displaying virtualinformation so as to be superimposed over an image obtained by imaging asubject (imaging target) in the real world.

For example, there is known a technique of identifying an imaging targeton the basis of the position and the angle of a terminal device, anddisplaying virtual information which has been previously associated withthe imaging target so as to be superimposed over an image of the imagingtarget.

However, since the association relationship between an imaging targetand virtual information is previously fixed in the conventionaltechnique, there has been no system capable of, for example, freelyregistering association data obtained by associating an image of animaging target with virtual information while imaging various imagingtargets. Therefore, the convenience of a user is disadvantageously low.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram illustrating an example of the configuration of aninformation processing system according to a first embodiment;

FIG. 2 is a diagram illustrating an example of the functionalconfiguration of a registration device according to the firstembodiment;

FIG. 3 is a diagram illustrating an example of the display on a screenaccording to the first embodiment;

FIG. 4 is a diagram illustrating an example of the display on the screenaccording to the first embodiment;

FIG. 5 is a diagram illustrating an example of the display on the screenaccording to the first embodiment;

FIG. 6 is a flowchart illustrating an example of the operation of theregistration device according to the first embodiment;

FIG. 7 is a diagram illustrating an example of the configuration of aserver device according to the first embodiment;

FIG. 8 is a diagram schematically illustrating data stored in a firststorage unit according to the first embodiment;

FIG. 9 is a diagram illustrating an example of the functionalconfiguration of a terminal device according to the first embodiment;

FIG. 10 is a flowchart illustrating an example of the operation of theterminal device according to the first embodiment;

FIG. 11 is a diagram illustrating an example of the display on a screenaccording to a modification;

FIG. 12 is a diagram illustrating an example of the functionalconfiguration of an information processing apparatus according to asecond embodiment;

FIG. 13 is a flowchart illustrating an example of the operation of theinformation processing apparatus according to the second embodiment;

FIG. 14 is a flowchart illustrating an example of the operation of theinformation processing apparatus according to the second embodiment; and

FIG. 15 is a diagram illustrating an example of the functionalconfiguration of an information processing apparatus according to amodification.

DETAILED DESCRIPTION

According to an embodiment, an information processing apparatus includesa first image acquisition unit, a first information acquisition unit, areception unit, a generation unit, and a writing unit. The first imageacquisition unit is configured to acquire a first image obtained byimaging by an imaging unit. The first information acquisition unit isconfigured to acquire any of multiple pieces of information capable ofbeing associated with the first image. The reception unit is configuredto receive a first input. The generation unit is configured to associatethe first image acquired in the first image acquisition unit with theinformation piece acquired in the first information acquisition unit togenerate an association data piece when the reception unit receives thefirst input. The writing unit is configured to write the associationdata piece into a storage unit.

Various embodiments will be described in detail below with reference tothe accompanying drawings.

First Embodiment

FIG. 1 is a diagram illustrating an example of the schematicconfiguration of an information processing system 1 according to thefirst embodiment. As illustrated in FIG. 1, the information processingsystem 1 includes a registration device 10, a server device 20, and aterminal device 30. In the example illustrated in FIG. 1, the serverdevice 20 is connected to each of the registration device 10 and theterminal device 30 via a network. In this specification, the informationprocessing system 1 including a single terminal device 30 will bedescribed as an example for convenience of explanation. However, thenumber of terminal devices 30 provided in the information processingsystem 1 is not limited to one, and may be any number.

First, the details of the registration device 10 will be described. FIG.2 is a diagram illustrating an example of the functional configurationof the registration device 10. The registration device 10 includes aportable information processing apparatus that has a function of imagingan imaging target (subject) and a touch panel function, and correspondsto “information processing apparatus” in the claims. In this example,the registration device 10 is equipped with a camera (not illustrated)which is an example of “imaging unit” in the claims and a touch paneldisplay (not illustrated). In the present specification, “imaging”indicates converting an image of a subject (imaging target) formed by anoptical system such as a lens into an electrical srgnal.

As illustrated in FIG. 2, the registration device 10 includes a cameracontrol unit 11, a first display unit 12, a first image acquisition unit13, a first information acquisition unit 14, a reception unit 15, ageneration unit 16, and a writing unit 17.

When starting using the camera mounted on the registration device 10,the camera control unit 11 continuously performs control for imaging animaging target at which a lens of the camera is pointed at apredetermined cycle. Every time when imaging is performed by the camera,the first display unit 12 performs control for displaying an imageobtained by the imaging (in the following description, may also bereferred to as “first image”) on the touch panel display mounted on theregistration device 10. The touch panel display mounted on theregistration device 10 has an area in which various pieces ofinformation are displayed (in the following description, referred to as“screen”), and the first image obtained by imaging by the camera isdisplayed on the screen. In this example, when an imaging target atwhich the lens of the camera is pointed is switched to another imagingtarget, or when a target over which the registration device 10 equippedwith the camera is held is switched to another target, a first imagedisplayed on the screen is also switched to another first image inresponse thereto.

In the present embodiment, the first display unit 12 performs controlfor displaying multiple pieces of content capable of being associatedwith (linked to) a first image on the screen. In this specification, thecontent indicates information that includes at least one of an image(video), voice, text and URL indicating a link destination (addressinformation), and corresponds to “information” in the claims. As will bedescribed later, in the present embodiment, content is stored in theserver device 20. The first display unit 12 accesses the server device20 to acquire multiple pieces of content capable of being associatedwith a first image. Then, the first display unit 12 performs control fordisplaying the acquired multiple pieces of content (or may also be iconimages corresponding to the respective pieces of content) on the screen.

FIG. 3 is a diagram illustrating an example of the display on the screenaccording to the first embodiment. As illustrated in FIG. 3, the screenincludes an image area 40 in which the first image is displayed and acontent area 50 in which multiple pieces of content are displayed. Inthis example, the image area 40 corresponds to “image area” in theclaims, and the content area 50 corresponds to “information area” in theclaims.

The description will be continued with reference to FIG. 2. The firstimage acquisition unit 13 has a function of acquiring the first image.In the present embodiment, the first image acquisition unit 13 has afunction of acquiring the first image displayed on the screen accordingto an instruction from the reception unit 15.

The first information acquisition unit 14 has a function of acquiringany of multiple pieces of content. In the present embodiment, the firstinformation acquisition unit 14 has a function of acquiring any ofmultiple pieces of content that are displayed on the content area 50according to an instruction from the reception unit 15.

The reception unit 15 has a function of receiving a first input from auser. In the present embodiment, the first input indicates an input forinstructing selection of any piece of the content. When receiving thefirst input, the reception unit 15 instructs the first image acquisitionunit 13 to acquire a first image, and instructs the first informationacquisition unit 14 to acquire a content piece that has been instructedto be selected.

More specifically, the first input in the present embodiment is an inputaccording to an operation of: bringing a pointing object such as afinger and a touch pen into contact with a first area on the screen inwhich any piece of the content is displayed; then moving the pointingobject to the image area 40 with being in contact with the screen; andthen releasing the pointing object from the screen. In this example, thereception unit 15 has a function of detecting the contact position(touch position) of the pointing object with the screen and thetrajectory of the contact position. The reception unit 15 instructs thefirst display unit 12 to display the content piece so as to follow thedetected contact position and trajectory. Upon receiving theinstruction, the first display unit 12 performs control for displayingthe content piece displayed in the first area so as to follow thetrajectory of the pointing object which moves while making contact withthe screen.

When receiving the first input, the reception unit 15 instructs thefirst image acquisition unit 13 to acquire a first image that is beingdisplayed on the screen when the pointing object is released from thescreen, and instructs the first information acquisition unit 14 toacquire a content piece corresponding to the first area (a content piecethat has been instructed to be selected).

When the reception unit 15 receives the first input, the generation unit16 associates the first image acquired in the first image acquisitionunit 13 with the content piece acquired in the first informationacquisition unit 14 to thereby generate an association data piece. Inthe present embodiment, when receiving the first input, the receptionunit 15 instructs the generation unit 16 to generate an association datapiece. Then, upon receiving the instruction, the generation unit 16associates the first image acquired in the first image acquisition unit13 (in this example, the first image that is being displayed when thepointing object is released from the screen) with the content pieceacquired in the first information acquisition unit 14 (the content piececorresponding to the first area) to thereby generate an association datapiece. In this example, the generation unit 16 associates image thatindicates the first image acquired in the first image acquisition unit13 with a content ID that identifies the content piece acquired in thefirst information acquisition unit 14 to thereby generate theassociation data piece. However, the generation unit 16 is not limitedthereto.

For example, a case where a user performs an operation in which a fingerof the user is brought into contact with the first area in which acontent piece X is displayed in the content area 50 as illustrated inFIG. 4, then moved to the image area 40 with being in contact with thescreen as illustrated in FIG. 5, and then released from the screen isassumed. In this case, as illustrated in FIGS. 4 and 5, the firstdisplay unit 12 performs control for displaying the content piece X soas to follow the trajectory of the finger. When a user releases his/herfinger from the screen as illustrated in FIG. 5, the first image that isbeing displayed in the image area 40 at this moment is associated withthe content piece X so that an association data piece is generated.

The description will be continued with reference to FIG. 2. The writingunit 17 performs control for writing the association data piecegenerated in the generation unit 16 into the server device 20. Although,in this example, the server device 20 corresponds to “storage unit” inthe claims, the server device 20 is not limited thereto.

The registration device 10 according to the present embodiment has ahardware configuration that uses a computer provided with a CPU, astorage unit such as read only memory (ROM) and RAM, and a touch paneldisplay. The functions of the respective units (the camera control unit11, the first display unit 12, the first image acquisition unit 13, thefirst information acquisition unit 14, the reception unit 15, thegeneration unit 16, and the writing unit 17) of the registration device10 are implemented by executing programs stored in the storage unit bythe CPU, but not limited thereto. For example, at least some of thefunctions of the respective units of the registration device 10 may alsobe implemented by a hardware circuit (a semiconductor integratedcircuit, for example).

Next, an example of the operation of the registration device 10 will bedescribed. FIG. 6 is a flowchart illustrating an example of theoperation of the registration device 10. As illustrated in FIG. 6, thereception unit 15 first determines whether or not the first input hasbeen received (step S1). When the first input is determined to have beenreceived (YES at step S1), the reception unit 15 instructs the firstimage acquisition unit 13 to acquire a first image, and instructs thefirst information acquisition unit 14 to acquire a content piece thathas been instructed to be selected. Further, the reception unit 15instructs the generation unit 16 to generate an association data piece.

Upon receiving the instruction from the reception unit 15, the firstimage acquisition unit 13 acquires the first image (step S2). Uponreceiving the instruction from the reception unit 15, the firstinformation acquisition unit 14 acquires the content piece that has beeninstructed to be selected (step S3). Step S2 and step S3 can beperformed in any order, and may also be performed at the same time.

Then, upon receiving the instruction from the reception unit 15, thegeneration unit 16 associates the first image acquired in step S2 withthe content piece acquired in step S3 to thereby generate an associationdata piece (step S4). Then, the writing unit 17 performs control forwriting the association data piece generated in step S4 into the serverdevice 20 (step S5).

Next, the details of the server device 20 will be described. Asillustrated in FIG. 7, the server device 20 includes a first storageunit 22 and a second storage unit 24. The first storage unit 22 storestherein association data generated in the registration device 10. FIG. 8is a diagram schematically illustrating multiple pieces of associationdata stored in the first storage unit 22. As illustrated in FIG. 8, thefirst storage unit 22 stores therein multiple pieces of association dataeach piece of which is generated by associating image indicating a firstimage with one or more content IDs. For example, an association datapiece in the first line indicates that a content ID representing“CONTENT 1” and a content ID representing “CONTENT 2” are associatedwith image representing “IMAGE A”.

The second storage unit 24 associates respective multiple content IDswith content information indicating respective content pieces(information including the contents and the displayed position), andstores therein the associated content IDs and content information. Inthis example, the associated content piece (content information) can beidentified by using the content ID contained in the association datapiece as a key. Therefore, the association data piece can be regarded asassociating image (first image) with a content piece. The configurationof each association data piece is not limited to the aboveconfiguration. For example, the association data piece may be configuredby associating image with content information.

Although not described in detail here, the server device 20 has afunction of communicating with each of the registration device 10 andthe terminal device 30. In response to a request from the registrationdevice 10 or the terminal device 30, the server device 20 can writeassociation date into the first storage unit 22, and read out (alsotransmit) and delete information stored in the first storage unit 22 andthe second storage unit 24.

Next, the details of the terminal device 30 will be described. FIG. 9 isa block diagram illustrating an example of the functional configurationof the terminal device 30. In the present embodiment, the terminaldevice 30 includes a portable information processing apparatus that hasa function of imaging an imaging target. In this example, the terminaldevice 30 is equipped with a camera (not illustrated). As illustrated inFIG. 9, the terminal device 30 includes a camera control unit 31, asecond image acquisition unit 32, a second information acquisition unit33, and a second display unit 34.

When starting using the camera mounted on the terminal device 30, thecamera control unit 31 continuously performs control for imaging animaging target at which a lens of the camera is pointed at apredetermined cycle. Every time when imaging is performed by the camera,the second image acquisition unit 32 acquires an image obtained by theimaging (in the following description, may also be referred to as“second image”), and outputs the acquired second image to the secondinformation acquisition unit 33.

The second information acquisition unit 33 determines whether or not thesecond image acquired by the second image acquisition unit 32 matchesany image (first image) contained in the association data registered inthe server device 20. When there is no image that matches the secondimage acquired by the second image acquisition unit 32 in the imagecontained in the association data registered in the server device 20,the second information acquisition unit 33 directly outputs the secondimage to the second display unit 34. On the other hand, when there is animage that matches the second image acquired by the second imageacquisition unit 32 in the image contained in the association dataregistered in the server device 20, the second information acquisitionunit 33 acquires a content piece that is associated with the image thatmatches the second image. In the present embodiment, the secondinformation acquisition unit 33 acquires content informationcorresponding to a content ID that is associated with the image thatmatches the second image in the image contained in the registeredassociation data from the server device 20. Then, the second informationacquisition unit 33 outputs the content piece (content information)acquired from the server device 20 to the second display unit 34together with the second image acquired by the second image acquisitionunit 32.

The second display unit 34 performs control for displaying theinformation output from the second information acquisition unit 33 on adisplay device (not illustrated) which is mounted on the terminal device30. The display device mounted on the terminal device 30 has an area(screen) on which various pieces of information are displayed, andinformation output from the second information acquisition unit 33 isdisplayed on the screen. As described above, for example, when there isno image that matches the second image acquired by the second imageacquisition unit 32 in the image contained in the registered associationdata, the second information acquisition unit 33 directly outputs thesecond image to the second display unit 34. Therefore, the seconddisplay unit 34 performs control for directly displaying the secondimage output from the second information acquisition unit 33 on thescreen.

On the other hand, when there is an image that matches the second imageacquired by the second image acquisition unit 32 in the image containedin the registered association data, the second information acquisitionunit 33 outputs a content piece that is associated with the image thatmatches the second image and the second image acquired by the secondimage acquisition unit 32. Therefore, the second display unit 34performs control for displaying a superimposed image of the second imageoutput from the second information acquisition unit 33 and the contentpiece on the screen.

The terminal device 30 according to the present embodiment has ahardware configuration that uses a computer provided with a CPU, astorage unit such as read only memory (ROM) and RAM, and a displaydevice. The functions of the respective units (the camera control unit31, the second image acquisition unit 32, the second informationacquisition unit 33, and the second display unit 34) of the terminaldevice 30 are implemented by executing programs stored in the storageunit by the CPU, but not limited thereto. For example, at least some ofthe functions of the respective units of the terminal device 30 may alsobe implemented by a hardware circuit (a semiconductor integratedcircuit, for example).

Next, an example of the operation of the terminal device 30 will bedescribed. FIG. 10 is a flowchart illustrating an example of theoperation of the terminal device 30. As illustrated in FIG. 10, everytime when imaging is performed by the camera, the second imageacquisition unit 32 acquires an image (second image) obtained by theimaging (step S11). Then, the second information acquisition unit 33determines whether or not there is any image that matches the secondimage acquired in step S11 in the image contained in the associationdata registered in the server device 20 (step S12).

In step S12 described above, when an image that matches the second imageacquired in step S11 is determined to exist in the image contained inthe registered association data (YES at step S12), the secondinformation acquisition unit 33 acquires a content piece that isassociated with the image that matches the second image acquired in stepS11 from the server device 20 (step S13). Then, the second informationacquisition unit 33 outputs the acquired content piece and the secondimage acquired in step S11 to the second display unit 34. Then, thesecond display unit 34 performs control for displaying a superimposedimage of the second image output from the second information acquisitionunit 33 and the content piece on the screen (step S14).

On the other hand, when an image that matches the second image acquiredin step S11 is determined not to exist in the image contained in theregistered association data (NO at step S12), the second informationacquisition unit 33 directly outputs the second image acquired in stepS11 to the second display unit 34. Then, the second display unit 34performs control for directly displaying the second image output fromthe second information acquisition unit 33 on the screen (step S15).

As described above, in the present embodiment, when receiving the firstinput, a first image obtained by imaging by the camera is associatedwith any piece of the content to generate (and register) an associationdata piece. Therefore, a user can freely register an association datapiece in which the first image is associated with any piece of thecontent by pointing the lens of the camera mounted on the registrationdevice 10 at a desired imaging target, or holding the registrationdevice 10 equipped with the camera over a desired imaging target toperform the operation for the first input.

More specifically, the first input is made for instructing selection ofany piece of the content. When receiving the first input, the receptionunit 15 instructs the first image acquisition unit 13 to acquire a firstimage, and instructs the first information acquisition unit 14 toacquire a content piece that has been instructed to be selected. In theabove embodiment, only by pointing the lens of the camera mounted on theregistration device 10 at a desired imaging target and performing anoperation for instructing selection of a desired content piece whileconfirming a first image displayed on the screen, a user can generate anassociation data piece in which the first image that is being displayedon the screen at this moment is associated with the content piece thathas been instructed to be selected, and register the generatedassociation data piece. As described above, according to the presentembodiment, an advantageous effect such as further improved convenienceof a user who registers association data can be achieved.

Modification 1 of First Embodiment

For example, the first input may be an input according to an operationof bringing a pointing object (a finger or a touch pen, for example)into contact with a first area on the screen in which any piece ofcontent is displayed. When receiving such a first input, the receptionunit 15 instructs the first image acquisition unit 13 to acquire a firstimage that is being displayed on the screen when the pointing objectmakes contact with the screen, and instructs the first informationacquisition unit 14 to acquire a content piece that corresponds to thefirst area (a content piece corresponding to an area on the screen withwhich the pointing object makes contact). In this case, only by bringingthe pointing object such as a finger into contact with the area (thefirst area) in which a desired content piece is displayed in the contentarea 50, a user can generate an association data piece by associatingthe first image that is being displayed on the screen at this momentwith the desired content piece and register the generated associationdata piece. As a result, the convenience of a user is further improved.

Modification 2 of First Embodiment

The first input is only required to be an initiator for generating anassociation data piece, and the mode thereof can therefore bevoluntarily changed. For example, the first input may include an inputfor instructing display of a still image of an imaging target at whichthe lens of the camera mounted on the registration device 10 is pointedand an input for instructing selection of any piece of content. In sucha first input, for example, when a user performs an operation forinstructing display of a still image of a first image that is beingcurrently displayed on the screen while confirming first images whichare sequentially displayed on the screen such as an operation oftouching an icon of a photographing button, the reception unit 15receives an input according to the operation. Then, the reception unit15 instructs the first display unit 12 to display the still image of thefirst image that is currently being displayed on the screen, andinstructs the first image acquisition unit 13 to acquire the first imagedisplayed as the still image. Thereafter, when the user performs anoperation for instructing selection of a desired content piece, thereception unit 15 receives an input according to the operation. Then,the reception unit 15 can instruct the first information acquisitionunit 14 to acquire the content piece that has been instructed to beselected, and instruct the generation unit 16 to generate an associationdata piece.

In this example, a user points the lens of the camera mounted on theregistration device 10 at a desired imaging target, or holds theregistration device 10 equipped with the camera over a desired imagingtarget, then performs an operation for instructing display of a stillimage of a first image that is being currently displayed on the screenwhile confirming first images which are sequentially displayed on thescreen such as an operation of touching an icon of a photographingbutton, and then performs an operation for instructing selection of adesired content piece. As a result, the user can generate an associationdata piece in which the still image of the first image is associatedwith the content piece that has been instructed to be selected, andregister the generated association data piece.

Modification 3 of First Embodiment

For example, when a first image acquired in the first image acquisitionunit 13 matches or is similar to an image contained in association dataregistered in the server device 20, and a content piece that isassociated with the image that matches or is similar to the first imageacquired in the first image acquisition unit 13 in content contained inthe association data registered in the server device 20 matches acontent piece acquired in the first information acquisition unit 14, thegeneration unit 16 may not generate an association data piece. That is,when attempting to associate the same content piece with similar images,overlapping registration may not be performed.

In such a configuration, a function of determining whether or not afirst image acquired in the first image acquisition unit 13 matches oris similar to an image contained in association data registered in theserver device 20 and a function of determining whether or not a contentpiece that is associated with the image that matches or is similar tothe first image acquired in the first image acquisition unit 13 incontent contained in the association data registered in the serverdevice 20 matches a content piece acquired in the first informationacquisition unit 14 may be provided in the generation unit 16, or mayalso be separately (independently) provided from the generation unit 16.

Modification 4 of First Embodiment

For example, as illustrated in FIG. 11, the first display unit 12 canalso display third images capable of identifying first images with whichrespective content pieces have already been associated. In the exampleillustrated in FIG. 11, the third images are thumbnail images (reducedimages for listing multiple images) of first images with whichrespective content pieces have already been associated. However, thethird images are not limited thereto. In the example illustrated in FIG.11, the screen further includes a candidate area 60 in which themultiple third images are displayed in addition to the image area 40 andthe content area 50.

In the example illustrated in FIG. 11, when receiving a sixth input forinstructing selection of any of the third images, the reception unit 15instructs the first image acquisition unit 13 to acquire a first image,and instructs the generation unit 16 to generate an association datapiece in which a content piece that corresponds to the third image thathas been instructed to be selected (the same content piece as a contentpiece that has already been associated with the first image identifiedby the selected third image) is associated with the first image acquiredin the first image acquisition unit 13. In this example, the sixth inputindicates an input according to an operation of bringing a pointingobject such as a finger and a touch pen into contact with an area inwhich any of the third images is displayed in the candidate area 60.However, the sixth input is not limited thereto.

For example, when attempting to associate multiple content pieces thatare the same as multiple content pieces associated with an imagecontained in the registered association data with an imaging target atwhich the lens of the camera mounted on the registration device 10 ispointed, it is only required for a user to perform an operation forinstructing selection of a thumbnail image (third image) thatcorresponds to the image (the first image with which the content piecesare already associated) in the candidate area 60. Therefore, theconvenience of a user is further improved.

Second Embodiment

Next, the second embodiment will be described. The second embodiment isdifferent from the first embodiment in that the registration device 10and the terminal device 30 are configured as a single device.Hereinbelow, the details of the second embodiment will be described. Adescription of overlapping points with the first embodiment willappropriately be omitted.

FIG. 12 is a diagram illustrating an example of the functionalconfiguration of an information processing apparatus 100 according tothe second embodiment. The information processing apparatus 100 includesa portable information processing apparatus that has a function ofimaging an imaging target and a touch panel function, and corresponds to“information processing apparatus” in the claims. In this example, theinformation processing apparatus 100 is equipped with a camera (notillustrated) and a touch panel display (not illustrated). Further, theinformation processing apparatus 100 is equipped with the functions ofthe registration device 10 and the functions of the terminal device 30.The information processing apparatus 100 is connected to the serverdevice 20 via a network.

As illustrated in FIG. 12, the information processing apparatus 100includes a first image acquisition unit 13, a first informationacquisition unit 14, a reception unit 15, a generation unit 16, awriting unit 17, a deletion unit 18, a camera control unit 31, a secondimage acquisition unit 32, a second information acquisition unit 33, anda second display unit 34.

In this example, the camera control unit 31 also serves as the cameracontrol unit 11 described above. The second display unit 34 also servesas the first display unit 12 described above. Therefore, the seconddisplay unit 34 also performs control for displaying multiple pieces ofcontent that can be associated with an image obtained by imaging by thecamera on the screen (here, the screen indicates an area in whichvarious pieces of information are displayed on the touch panel displaymounted on the information processing apparatus 100). In the followingdescription, an area on the screen in which an image (including asuperimposed image) is displayed is referred to as an image area, and anarea on the screen in which multiple pieces of content (may be iconimages corresponding to the respective pieces of content) are displayedis referred to as a content area. In this example, an image obtained byimaging by the camera mounted on the information processing apparatus100 can be regarded as the above-described second image, and can also beregarded as the above-described first image. In the followingdescription, in images obtained by imaging by the camera, an imageacquired in the second image acquisition unit 32 is referred to as“second image”, and an image acquired in the first image acquisitionunit 13 is referred to as “first image”, thereby distinguishing theimages.

In the present embodiment, when a user starts using the camera mountedon the information processing apparatus 100, the camera control unit 31continuously performs control for imaging an imaging target (subject) atwhich a lens of the camera is pointed at a predetermined cycle. Everytime when imaging is performed by the camera, the second imageacquisition unit 32 acquires an image obtained by the imaging, andoutputs the acquired image (second image) to the second informationacquisition unit 33. Then, in the same manner as in the firstembodiment, when there is no image that matches the second imageacquired by the second image acquisition unit 32 in image contained inassociation data registered in the server device 20, the second image isdirectly displayed on the screen. On the other hand, when there is animage that matches the second image in the image contained in theassociation data registered in the server device 20, a superimposedimage of a content piece that is associated with the image that matchesthe second image and the second image is displayed on the screen.

Next, the function of the deletion unit 18 will be described. Under thestate that the superimposed image is displayed on the screen, when thereception unit 15 receives a second input for instructing deletion of anassociation data piece in which a content piece included in asuperimposed image is associated with an image, the deletion unit 18performs control for deleting the association data piece from the serverdevice 20. The second input in the present embodiment indicates an inputaccording to an operation of: bringing a pointing object into contactwith a second area in which the content piece included in thesuperimposed image is displayed in the image area; moving the pointingobject to the outside of the image area (to the content area, forexample) with being in contact with the screen; and then releasing thepointing object from the screen. However, the second input is notlimited thereto. For example, the second input may be an input accordingto an operation of bringing a pointing object such as a finger and atouch pen into contact with an icon (UI image) which is used forinstructing deletion of the association data piece that corresponds tothe content piece included in the superimposed image and displayed onthe screen. Further, the second input may be an input according to anoperation of shaking the information processing apparatus 100.

When the reception unit 15 receives the second input, the reception unit15 instructs the deletion unit 18 to delete the association data piecein which the content piece included in the superimposed image isassociated with the image. Upon receiving the instruction, the deletionunit 18 performs control for deleting the association data piece inwhich the content piece included in the superimposed image is associatedwith the image from the server device 20. More specifically, thedeletion unit 18 performs control for deleting the association datapiece in which a content ID that identifies the content piece includedin the superimposed image is associated with the image among theassociation data pieces registered in the server device 20 from thefirst storage unit 22.

FIG. 13 is a flowchart illustrating an example of the operation of theinformation processing apparatus 100 when deleting an association datapiece that corresponds to a content piece included in a superimposedimage. As illustrated in FIG. 13, the reception unit 15 first determineswhether or not the second input has been received (step S21). When thesecond input is determined to have been received (YES at step S21), thereception unit 15 instructs the deletion unit 18 to delete anassociation data piece that corresponds to a content piece included in asuperimposed image. Upon receiving the instruction, the deletion unit 18performs control for deleting the association data piece thatcorresponds to the content piece included in the superimposed image fromthe server device 20 (step S22).

Next, an example of a method for generating an association data piece inwhich a content piece included in a superimposed image is associatedwith a new first image and registering the generated association datapiece. In the present embodiment, when the reception unit 15 receives athird input for instructing registration of an association data piece inwhich a content piece included in a superimposed image is associatedwith a first image newly acquired in the first image acquisition unit13, the generation unit 16 generates the association data piece in whichthe content piece included in the superimposed image is associated withthe first image newly acquired in the first image acquisition unit 13.Then, the writing unit 17 performs control for writing the associationdata piece newly generated by the generation unit 16 into the serverdevice 20.

The third input in the present embodiment includes a fourth input forinstructing selection of a content piece that is included in asuperimposed image and a fifth input for instructing acquisition of afirst image. When receiving the fourth input, the reception unit 15selects the content piece included in the superimposed image. In thepresent embodiment, the fourth input indicates an input according to anoperation of bringing a pointing object into contact with the secondarea on the screen in which the content piece included in thesuperimposed image is displayed. However, the fourth input is notlimited thereto. For example, the fourth input may be an input accordingto an operation of shaking the information processing apparatus 100, andmay also be an input according to an operation of lifting up andscooping the information processing apparatus 100. When receiving thefourth input, the reception unit 15 selects the content piece that isincluded in the superimposed image.

A method for selecting a content piece included in a superimposed imagemay be any method. For example, the reception unit 15 can select acontent piece included in a superimposed image by instructing deletionunit 18 to delete an association data piece that corresponds to thecontent piece included in the superimposed image and also cutting off acontent ID from the association data piece that corresponds to thecontent piece included in the superimposed image. Alternatively, forexample, the reception unit 15 can select a content piece included in asuperimposed image by generating a copy of a content ID from anassociation data piece that corresponds to the content piece included inthe superimposed image.

Further, when receiving the fourth input, the reception unit 15instructs the second display unit 34 to perform control for displayingthe content piece included in the superimposed image so as to stick to apointing object. Upon receiving the instruction, the second display unit34 performs control for displaying the content piece included in thesuperimposed image so as to stick to the pointing object. As a result, auser can intuitively understand that the content piece included in thesuperimposed image has been selected.

When receiving a fifth input after the fourth input, the reception unit15 instructs the first image acquisition unit 13 to acquire a firstimage, and instructs the generation unit 16 to generate an associationdata piece in which the first image acquired in the first imageacquisition unit 13 is associated with the content piece that has beenselected in response to the fourth input. Although the fifth inputindicates an input according to an operation of releasing a pointingobject from the screen in the present embodiment, the fifth input is notlimited thereto. Upon receiving the instruction from the reception unit15, the first image acquisition unit 13 acquires an image that has beenobtained by imaging and is being displayed on the screen when thepointing object is released from the screen. Upon receiving theinstruction from the reception unit 15, the generation unit 16associates the image (first image) acquired by the first imageacquisition unit 13 with the content piece (the content ID in thisexample) selected by the reception unit 15 to thereby generate anassociation data piece.

For example, a user performs an operation of bringing a pointing objectinto contact with the second area on the screen in which a content pieceis displayed (an operation for instructing selection of the contentpiece), and then performs and operation of switching an imaging target(a target over which the information processing apparatus 100 equippedwith the camera is held) to another imaging target with the pointingobject being in contact with the screen to thereby switch an imagedisplayed on the screen to another image and then releasing the pointingobject from the screen. In this manner, a user can generate anassociation data piece in which the content piece that has beeninstructed to be selected is associated with the image that is beingdisplayed when the pointing object is released from the screen, andregister the generated association data piece.

FIG. 14 is a flowchart illustrating an example of the operation of theinformation processing apparatus 100 when generating an association datapiece in which a content piece included in a superimposed image isassociated with a newly acquired first image, and registering thegenerated association data piece. As illustrated in FIG. 14, thereception unit first determines whether or not the fourth input has beenreceived (step S31). When the fourth input is determined to have beenreceived (YES at step S31), the reception unit 15 selects a contentpiece included in a superimposed image (step S32). Then, the receptionunit 15 determines whether or not the fifth input has been received(step S33). When the fifth input is determined to have been received(YES at step S33), the reception unit 15 instructs the first imageacquisition unit 13 to acquire a first image, and instructs thegeneration unit 16 to generate an association data piece in which thefirst image acquired in the first image acquisition unit 13 isassociated with the content piece selected in step S32. Upon receivingthe instruction from the reception unit 15, the first image acquisitionunit 13 acquires an image that has been obtained by imaging and is beingdisplayed on the screen when a pointing object is released from thescreen. Upon receiving the instruction from the reception unit 15, thegeneration unit 16 associates the image (first image) acquired by thefirst image acquisition unit 13 with the content piece selected in stepS32 to thereby generate an association data piece (step S34). Then, thewriting unit 17 performs control for writing the association data piecegenerated in step S34 into the server device 20 (step S35).

Also in the second embodiment, the same effect as in the firstembodiment can be obtained.

Modification 1 of Second Embodiment

For example, under the state that a superimposed image is displayed onthe screen, when a new content piece is selected, the new content piecemay be added to the registered association data. FIG. 15 is a blockdiagram illustrating an example of the functional configuration of aninformation processing apparatus 1000 according to Modification 1. Asillustrated in FIG. 15, the image processor 1000 is different from thesecond embodiment in that the image processor 1000 further includes anupdate unit 70.

In the example illustrated in FIG. 15, when the reception unit 15receives an input for instructing selection of any of multiple pieces ofcontent that are displayed on the content area (corresponding to thefirst input described above), with a superimposed image displayed on thescreen, the reception unit 15 instructs the first informationacquisition unit 14 to acquire the content piece that has beeninstructed to be selected. Further, the reception unit 15 instructs theupdate unit 70 to update an association data piece (an association datapiece registered in the server device 20) that corresponds to a contentpiece included in the superimposed image.

Upon receiving the instruction from the reception unit 15, the updateunit 70 performs control for adding the content piece acquired in thefirst information acquisition unit 14 to the registered association datapiece that corresponds to the content piece included in the superimposedimage. As a result, the registered association data piece thatcorresponds to the content piece included in the superimposed image isupdated into an association data piece in which the content pieceincluded in the superimposed image, the newly selected content piece(the content piece acquired in the first information acquisition unit14) and an image are associated with each other.

Modification 2 of Second Embodiment

For example, the first storage unit 22 and the second storage unit 24may be provided within the information processing apparatus 100 withoutproviding the server device 20.

Further, programs executed in the registration device 10, the terminaldevice 30 and the information processing apparatus 100 may be providedby being stored in a computer that is connected to a network such as theInternet, and causing the stored programs to be downloaded via thenetwork. Furthermore, the programs executed in the registration device10, the terminal device 30 and the information processing apparatus 100may also be provided or distributed via a network such as the Internet.Moreover, the programs executed in the registration device 10, theterminal device 30 and the information processing apparatus 100 may alsobe provided by being previously embedded in a nonvolatile recordingmedium such as ROM.

While certain embodiments have been described, these embodiments havebeen presented by way of example only, and are not intended to limit thescope of the inventions. Indeed, the novel embodiments described hereinmay be embodied in a variety of other forms; furthermore, variousomissions, substitutions and changes in the form of the embodimentsdescribed herein may be made without departing from the spirit of theinventions. The accompanying claims and their equivalents are intendedto cover such forms or modifications as would fall within the scope andspirit of the inventions.

What is claimed is:
 1. An information processing apparatus comprising: ahardware processor configured to control the apparatus to at least:acquire a first image obtained by imaging; acquire any of multiplepieces of information capable of being associated with the first image;receive a first input; associate the acquired first image with theacquired information piece to generate an association data piece whenthe first input is received; write the association data piece intostorage; and control displaying of the first image and one or morepieces of the information, wherein the first input is an input based on:receiving selection by a pointing object with respect to a first area ofa screen that includes an image area in which the first image isdisplayed and an information area in which one or more pieces of theinformation are displayed; detecting moving of the pointing object tothe image area and then detecting releasing of the pointing object, andwhen the first input is received the first image displayed on the screenwhen the pointing object is released is acquired, and the informationpiece corresponding to the first area is acquired.
 2. The apparatusaccording to claim 1, wherein the first input is an input based on:detecting the pointing object in contact with the first area of thescreen; detecting moving of the pointing object to the image area whilecontact with the screen is maintained; and then detecting releasing ofthe pointing object from the screen.
 3. The apparatus according to claim2, wherein the hardware processor is further configured to control theapparatus to display the information piece displayed in the first areaso as to follow a trajectory of the moving pointing object.
 4. Theapparatus according to claim 2, wherein the hardware processor isconfigured to control display of second images capable of identifyingfirst images with which information pieces have already been associated,and when receiving a second input for selecting any of the secondimages, the hardware processor is configured to control the apparatus toacquire the first image, and generate an association data piece in whichan information piece that corresponds to the second image that has beeninstructed to be selected is associated with the acquire first image. 5.The apparatus according to claim 1, wherein the hardware processor isfurther configured to control the apparatus to: acquire a second imageobtained by imaging; acquire an information piece associated with afirst image that matches the second image among first images containedin association data written into the storage; and display a superimposedimage of the second image and the acquired information piece associatedwith the matched first image.
 6. The apparatus according to claim 5,wherein the hardware processor is further configured to control theapparatus to delete from the storage the association data piece in whichthe information piece included in the superimposed image is associatedwith the matched first image, in response to receiving a second input.7. The apparatus according to claim 6, wherein the second input is aninput based on: detecting the pointing object in contact with a secondarea in which the information piece included in the superimposed imageis displayed on a screen that includes an image area in which thesuperimposed image is displayed; detecting moving of the pointing objectto outside the image area while contact with the screen is maintained;and then detecting releasing the pointing object from the screen, andwhen receiving the second input, the hardware processor is furtherconfigured to control the apparatus to delete, from the storage, theassociation data piece in which the information piece included in thesuperimposed image is associated with the matched first image.
 8. Theapparatus according to claim 5, wherein, when a second input is receivedfor instructing registration of an association data piece in which theinformation piece included in the superimposed image is associated witha newly acquired first image, the hardware processor is furtherconfigured to control the apparatus to generate an association datapiece in which the information piece included in the superimposed imageis associated with the newly acquired first image.
 9. The apparatusaccording to claim 8, wherein the second input includes a third inputfor selecting the information piece included in the superimposed imageand a fourth input for acquiring the new first image, and the hardwareprocessor is further configured to control the apparatus to select theinformation piece included in the superimposed image when receiving thethird input, and then acquire the new first image and generate theassociation data piece in which the newly acquired first image isassociated with the information piece selected in response to the thirdinput when receiving the fourth input.
 10. The apparatus according toclaim 9, wherein the third input is an input according to an operationof detecting a pointing object in contact with a second area in whichthe information piece included in the superimposed image is displayed ona screen that includes an image area in which the superimposed image isdisplayed, and the fourth input is an input according to an operation ofdetecting releasing of the pointing object from the screen.
 11. Theapparatus according to claim 10, wherein, when the third input isreceived, the hardware processor is configured to control the apparatusto display the information piece included in the superimposed image soas to stick to the pointing object.
 12. The apparatus according to claim1, wherein, when the acquired first image matches or is similar to afirst image contained in association data written into the storage, andwhen, among the information pieces contained in the association datawritten into the storage, the information piece associated with a firstimage that matches or is similar to the acquired first image matches theacquired information piece, the hardware processor is configured tocontrol the apparatus to not generate an association data piece.
 13. Theapparatus according to claim 1, wherein the hardware processor isconfigured to control the apparatus to display second images capable ofidentifying first images with which information pieces have already beenassociated, and when receiving a second input for selecting any of thesecond images, the hardware processor is configured to control theapparatus to acquire the first image, and to generate an associationdata piece in which an information piece that corresponds to the secondimage that has been instructed to be selected is associated with theacquired first image.
 14. An information processing method comprising:acquiring, by a processor, a first image obtained by imaging; acquiring,by the processor, any of multiple pieces of information capable of beingassociated with the first image; generating, by the processor, anassociation data piece by associating the acquired first image with theacquired information piece when receiving a first input; writing, by theprocessor, the generated association data piece into a storage unit; andcontrolling, by the processor, of display of the first image and one ormore pieces of the information, wherein the first input is an inputbased on: receiving selection by a pointing object with respect to afirst area of a screen that includes an image area in which the firstimage is displayed and an information area in which one or more piecesof the information are displayed; detecting moving of the pointingobject to the image area and then detecting releasing of the pointingobject; and when receiving the first input, acquiring the first imagethat is displayed on the screen when the pointing object is released,and acquiring the information piece corresponding to the first area. 15.A computer program product comprising a non-transitory computer-readablemedium containing a computer program embodied therein that, whenexecuted, causes a computer to control an apparatus to at least: acquirea first image obtained by imaging; acquire any of multiple pieces ofinformation capable of being associated with the first image; receive afirst input; associate the acquired first image with the acquiredinformation piece to generate an association data piece when the firstinput is received; write the association data piece into storage; andcontrol display of the first image and one or more pieces of theinformation, wherein the first input is an input based on: receivingselection by a pointing object with respect to a first area of a screenthat includes an image area in which the first image is displayed and aninformation area in which one or more pieces of the information aredisplayed; detecting moving of the pointing object to the image area andthen detecting releasing of the pointing object, and when receiving thefirst input, the first image that is displayed on the screen when theselection is released is acquired, and the information piececorresponding to the first area is acquired.